A search for rapidly modulated emission in bright X-ray sources using the HEAO A-1 data base

A search was performed in the HEAO A-1 Data Base (located at the Naval Research Laboratory in Washington, D.C.) for evidence of rapidly-rotating neutron stars that could be sources of coherent gravitational radiation. A new data analysis algorithm, which was developed, is described. The algorithm was applied to data from observations of Cyg X-2, Cyg X-3, and 1820-30. Upper limits on pulse fraction were derived and reported

Velocity measurements by laser resonance fluorescence

The photonburst correlation method was used to detect single atoms in a buffer gas. Real time flow velocity measurements with laser induced resonance fluorescence from single or multiple atoms was demonstrated and this method was investigated as a tool for wind tunnel flow measurement. Investigations show that single atoms and their real time diffusional motion on a buffer gas can be measured by resonance fluorescence. By averaging over many atoms, flow velocities up to 88 m/s were measured in a time of 0.5 sec. It is expected that higher flow speeds can be measured and that the measurement time can be reduced by a factor of 10 or more by careful experimental design. The method is clearly not ready for incorporation in high speed wind tunnels because it is not yet known whether the stray light level will be higher or lower, and it is not known what detection efficiency can be obtained in a wind tunnel situation

A Comparison of Learning Speed and Ability to Cope Without Exploration between DHP and TD(0)

This paper demonstrates the principal motivations for Dual Heuristic Dynamic Programming (DHP) learning methods for use in Adaptive Dynamic Programming and Reinforcement Learning, in continuous state spaces: that of automatic local exploration, improved learning speed and the ability to work without stochastic exploration in deterministic environments. In a simple experiment, the learning speed of DHP is shown to be around 1700 times faster than TD(0). DHP solves the problem without any exploration, whereas TD(0) cannot solve it without explicit exploration. DHP requires knowledge of, and differentiability of, the environment's model functions. This paper aims to illustrate the advantages of DHP when these two requirements are satisfied

Efficient Calculation of the Gauss-Newton Approximation of the Hessian Matrix in Neural Networks

The Levenberg-Marquardt (LM) learning algorithm is a popular algorithm for training neural networks; however, for large neural networks, it becomes prohibitively expensive in terms of running time and memory requirements. The most time-critical step of the algorithm is the calculation of the Gauss-Newton matrix, which is formed by multiplying two large Jacobian matrices together. We propose a method that uses back-propagation to reduce the time of this matrix-matrix multiplication. This reduces the overall asymptotic running time of the LM algorithm by a factor of the order of the number of output nodes in the neural network

Value-Gradient Learning

We describe an Adaptive Dynamic Programming algorithm VGL(λ) for learning a critic function over a large continuous state space. The algorithm, which requires a learned model of the environment, extends Dual Heuristic Dynamic Programming to include a bootstrapping parameter analogous to that used in the reinforcement learning algorithm TD(λ). We provide on-line and batch mode implementations of the algorithm, and summarise the theoretical relationships and motivations of using this method over its precursor algorithms Dual Heuristic Dynamic Programming and TD(λ). Experiments for control problems using a neural network and greedy policy are provided

To perform a gyro test of general relativity in a satellite and develop associated control technology

Performance tests of gyroscope operations and gyroscope readout equipment are discussed. The gyroscope was tested for 400 hours at liquid helium temperatures with spin speeds up to 30 Hz. Readout by observing trapped magnetic flux in the spinning rotor with a sensitive magnetometer was accomplished. Application of the gyroscope to space probes and shuttle vehicles

Neural-Network Vector Controller for Permanent-Magnet Synchronous Motor Drives: Simulated and Hardware-Validated Results

This paper focuses on current control in a permanentmagnet synchronous motor (PMSM). The paper has two main objectives: The first objective is to develop a neural-network (NN) vector controller to overcome the decoupling inaccuracy problem associated with conventional PI-based vector-control methods. The NN is developed using the full dynamic equation of a PMSM, and trained to implement optimal control based on approximate dynamic programming. The second objective is to evaluate the robust and adaptive performance of the NN controller against that of the conventional standard vector controller under motor parameter variation and dynamic control conditions by (a) simulating the behavior of a PMSM typically used in realistic electric vehicle applications and (b) building an experimental system for hardware validation as well as combined hardware and simulation evaluation. The results demonstrate that the NN controller outperforms conventional vector controllers in both simulation and hardware implementation

Clipping in Neurocontrol by Adaptive Dynamic Programming

In adaptive dynamic programming, neurocontrol, and reinforcement learning, the objective is for an agent to learn to choose actions so as to minimize a total cost function. In this paper, we show that when discretized time is used to model the motion of the agent, it can be very important to do clipping on the motion of the agent in the final time step of the trajectory. By clipping, we mean that the final time step of the trajectory is to be truncated such that the agent stops exactly at the first terminal state reached, and no distance further. We demonstrate that when clipping is omitted, learning performance can fail to reach the optimum, and when clipping is done properly, learning performance can improve significantly. The clipping problem we describe affects algorithms that use explicit derivatives of the model functions of the environment to calculate a learning gradient. These include backpropagation through time for control and methods based on dual heuristic programming. However, the clipping problem does not significantly affect methods based on heuristic dynamic programming, temporal differences learning, or policy-gradient learning algorithms

An Equivalence Between Adaptive Dynamic Programming With a Critic and Backpropagation Through Time

We consider the adaptive dynamic programming technique called Dual Heuristic Programming (DHP), which is designed to learn a critic function, when using learned model functions of the environment. DHP is designed for optimizing control problems in large and continuous state spaces. We extend DHP into a new algorithm that we call Value-Gradient Learning, VGL(λ), and prove equivalence of an instance of the new algorithm to Backpropagation Through Time for Control with a greedy policy. Not only does this equivalence provide a link between these two different approaches, but it also enables our variant of DHP to have guaranteed convergence, under certain smoothness conditions and a greedy policy, when using a general smooth nonlinear function approximator for the critic. We consider several experimental scenarios including some that prove divergence of DHP under a greedy policy, which contrasts against our proven-convergent algorithm